Skip to content

Fix: System prompts for Guardrailing Excerpt & Title#440

Merged
dkotter merged 1 commit intoWordPress:developfrom
Takshil-Kunadia:fix/issue-395-399-guardrails-on-excerpt-and-title
Apr 20, 2026
Merged

Fix: System prompts for Guardrailing Excerpt & Title#440
dkotter merged 1 commit intoWordPress:developfrom
Takshil-Kunadia:fix/issue-395-399-guardrails-on-excerpt-and-title

Conversation

@Takshil-Kunadia
Copy link
Copy Markdown
Contributor

@Takshil-Kunadia Takshil-Kunadia commented Apr 16, 2026

What?

Closes #395
Closes #399

It adds an output-format guardrail to the Excerpt Generation and Title Generation system instructions so the model returns only the excerpt or title and no conversational preamble, wrapper quotes, code fences, markdown, or trailing meta-commentary.

Why?

Both issues report the same class of bug: generated excerpts and titles arrive with extra wrapper text that gets saved verbatim into the post's excerpt / title / slug.

The root cause in both cases is that the existing system instructions do not explicitly constrain the model to return only the excerpt/title text. Frontier models mostly infer this; smaller instruction-tuned models do not, and add conversational scaffolding by default.

As discussed on #395, the agreed approach is a minimal prompt-level fix rather than output post-processing, and without bloating the instruction.

How?

Adds one requirement bullet at the end of each affected system instruction. The wording follows "eliminating preambles" recipes from major AI providers along with a positive output contract plus a short specific negative list which is the pattern empirically most robust across both frontier and smaller models.

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Opus 4.6
Used for: Researching cross-model prompt techniques for suppressing preambles (Anthropic prompt-engineering docs, OpenAI GPT-4.1 guide, Meta Llama guide). Final wording, file edits, and testing were implemented by me.

Testing Instructions

  1. Check out this branch.
  2. Configure a small/light model likely to exhibit the bug. Example: HuggingFace free-tier via the HuggingFace connector as described by the reported with Meta-Llama-3-8B-Instruct is what I tested with.
  3. Create or open a post with ~500 words.
  4. For Excerpt: in the sidebar, click Generate excerpt. Repeat 5–10 times. Expected: each generated excerpt is plain text with no Here's... / Sure, preface, no surrounding quotes, no markdown.
  5. For Title: click Re-generate. Repeat 5–10 times. Expected: a plain, single-line title; no Based on the content..., no ** / ###, no (N characters) suffix, no Why this works: explanation.
  6. Sanity check with a frontier model (OpenAI / Anthropic / Google) behavior should be unchanged from before this PR (the new bullet is not supposed to regress well-behaved models).

Screenshots or screencast

Before After
image image

Changelog Entry

Fixed - Excerpt and Title generation no longer include conversational preambles, wrapper quotes, markdown, or meta-commentary when using smaller language models.

Open WordPress Playground Preview

@github-actions
Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Unlinked Accounts

The following contributors have not linked their GitHub and WordPress.org accounts: @sagardholakiya, @AasthaPandya, @divyawpdev.

Contributors, please read how to link your accounts to ensure your work is properly credited in WordPress releases.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Unlinked contributors: sagardholakiya, AasthaPandya, divyawpdev.

Co-authored-by: Takshil-Kunadia <takshil@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>
Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: PriyankaGusani <priyankagusani@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.44%. Comparing base (219f424) to head (4a222ad).
⚠️ Report is 10 commits behind head on develop.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop     #440   +/-   ##
==========================================
  Coverage      68.44%   68.44%           
  Complexity       846      846           
==========================================
  Files             56       56           
  Lines           4095     4095           
==========================================
  Hits            2803     2803           
  Misses          1292     1292           
Flag Coverage Δ
unit 68.44% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jeffpaul jeffpaul added this to the 0.8.0 milestone Apr 16, 2026
@jeffpaul jeffpaul requested review from dkotter and jeffpaul April 16, 2026 21:32
@dkotter dkotter merged commit ca86475 into WordPress:develop Apr 20, 2026
18 checks passed
@jeffpaul
Copy link
Copy Markdown
Member

@sagardholakiya, @AasthaPandya, and @divyawpdev note that if you link your GitHub and WPORG profiles, then I can properly credit you in the AI plugin 0.8.0 release post as well as get the AI Contributor badge added to your WPORG profile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment